THEMIS: A Framework for Cost-Benefit Analysis of COVID-19 Non-Pharmaceutical Interventions (preprint)

Since December 2019, the world has been ravaged by the COVID-19 pandemic, with over 150 million confirmed cases and 3 million confirmed deaths worldwide. To combat the spread of COVID-19, governments have issued unprecedented non-pharmaceutical interventions (NPIs), ranging from mass gathering restrictions to complete lockdowns. Despite their proven effectiveness in reducing virus transmission, the policies often carry significant economic and humanitarian cost, ranging from unemployment to depression, PTSD, and anxiety. In this paper, we create a data-driven system dynamics framework, THEMIS, that allows us to compare the costs and benefits of a large class of NPIs in any geographical region across different cost dimensions. As a demonstration, we analyzed thousands of alternative policies across 5 countries (United States, Germany, Brazil, Singapore, Spain) and compared with the actual implemented policy. Our results show that moderate NPIs (such as restrictions on mass gatherings) usually produce the worst results, incurring significant cost while unable to sufficiently slow down the pandemic to prevent the virus from becoming endemic. Short but severe restrictions (complete lockdown for 4-5 weeks) generally produced the best results for developed countries, but only if the speed of reopening is slow enough to prevent a resurgence. Developing countries exhibited very different trade-off profiles from developed countries, and suggests that severe NPIs such as lockdowns might not be as suitable for developing countries in general.

National and subnational short-term forecasting of COVID-19 in Germany and Poland, early 2021 (preprint)

We report on the second and final part of a pre-registered forecasting study on COVID-19 cases and deaths in Germany and Poland. Fifteen independent research teams provided forecasts at lead times of one through four weeks from January through mid-April 2021. Compared to the first part (October--December 2020), the number of participating teams increased, and a number of teams started providing subnational-level forecasts. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in the first part of our study. In both countries, case counts declined initially, before rebounding due to the rise of the B.1.1.7 variant. Deaths declined through most of the study period in Germany while in Poland they increased after a prolonged plateau. Many, though not all, models outperformed a simple baseline model up to four weeks ahead, with ensemble methods showing very good relative performance. Major trend changes in reported cases, however, remained challenging to predict.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US (preprint)

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. In 2020, the COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized hundreds of thousands of specific predictions from more than 50 different academic, industry, and independent research groups. This manuscript systematically evaluates 23 models that regularly submitted forecasts of reported weekly incident COVID-19 mortality counts in the US at the state and national level. One of these models was a multi-model ensemble that combined all available forecasts each week. The performance of individual models showed high variability across time, geospatial units, and forecast horizons. Half of the models evaluated showed better accuracy than a naive baseline model. In combining the forecasts from all teams, the ensemble showed the best overall probabilistic accuracy of any model. Forecast accuracy degraded as models made predictions farther into the future, with probabilistic accuracy at a 20-week horizon more than 5 times worse than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. f

Short-term forecasting of COVID-19 in Germany and Poland during the second wave - a preregistered study (preprint)

We report insights from ten weeks of collaborative COVID-19 forecasting for Germany and Poland (12 October - 19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.